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Abstract 

Introduction: Efforts to implement and take to scale highly efficacious, low-cost interventions to prevent mother-to-child HIV 
transmission (pMTCT) have been a cornerstone of reproductive health services in sub-Saharan Africa for over a decade. Yet efforts 
to increase access and utilization of these services remain far from optimal. This study developed and applied an approach to 
systematically classify pMTCT performance to identify modifiable health system factors associated with pMTCT performance 
which may be replicated in other pMTCT systems. 

Methods: Facility-level performance measures were collected at 30 sites over a 12-month period and reviewed for consistency. 
Five combinations of three indicators (1. HIV testing; 2. CD4 testing; 3. a nti retroviral prophylaxis and combined a nti retroviral 
therapy initiation) were compared including a composite of all three, a combination of 1. and 3., and each individually. Approaches 
were visually assessed to describe facility performance, focusing on rank order consistency across high, medium and low cate- 
gories. Modifiable and non-modifiable factors were ascertained at each site and ranking process was reviewed to estimate 
association with facility performance through unadjusted Chi-square tests and logistic regression. After describing factors 
associated with high versus low performing pMTCT clinics, the effect of inclusion of the 10 middle performers was assessed. 
Results: The indicator most consistently associated with the reference composite indicator (HIV testing, a nti retroviral prophylaxis 
and combined a nti retroviral therapy) was the single measure of a nti retroviral prophylaxis and combined a nti retroviral therapy. 
Lower performing pMTCT clinics ranked consistently low across measurement strategies; high and middle performing clinics 
demonstrated more variability. Association between clinic characteristics and high pMTCT performance varied markedly across 
ranking strategies. Using the reference composite indicator, larger catchment area, higher number of institutional deliveries, 
onsite CD4 point-of-care capacity, and higher numbers of nurses and doctors were associated with high clinic performance while 
clinic location, NGO support, women's support group, community linkages patient-tracking systems and stock-outs were not 
associated with high performance. 

Conclusions: Classifying high and low performance provided consistent results across ranking measures, though granularity was 
improved by aggregating middle performers with either high or low performers. Human resources, catchment size and utilization 
were positively associated with effective pMTCT service delivery. 

Keywords: implementation science; performance measures; PMTCT; Mozambique; performance classification; health systems 
strengthening. 
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Introduction 

Sub-Saharan Africa bears a disproportionate HIV burden with 
92% of HIV-positive pregnant women and 90% of all incident 
HIV cases in children globally [1]. Efforts to implement and 
take to scale highly efficacious, low-cost interventions to 
prevent mother-to-child HIV transmission (pMTCT) have been 
a cornerstone of reproductive health services in high-burden 



HIV settings for over a decade, leading to substantial in- 
creases in access and utilization of preventive services, 
with the objective of prevention of paediatric HIV infection. 
However, pMTCT services in sub-Saharan Africa remain sub- 
optimal, with an estimated 56% of HIV-positive pregnant 
women accessing pMTCT services in 2012 [2]. Parallel to 
efforts to expand pMTCT, efforts have focused on implementing 
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more efficacious a nti retroviral (ARV) prophylaxis (PPO) regi- 
mens, which together have reduced HIV transmission to 
newborns [3,4]. 

To translate more efficacious regimens to effective pMTCT 
services, heterogeneous clinic-level facilitators and barriers 
that impact access and utilization must be addressed. By iden- 
tifying modifiable health system factors associated with high 
and low-performing pMTCT systems, best practices can be 
disseminated and implemented to maximize the number of 
women and children reached with effective pMTCT services. 
Likewise, identified factors associated with poorer per- 
formance can be addressed across facilities to reduce their 
replication. 

PMTCT is a highly complex package of healthcare interven- 
tions that spans multiple stages of women's reproductive 
and children's developmental lifespan, and health services 
specific to these stages. To maximize the benefit to mothers 
and children, pMTCT is delivered in a timed sequence, or 
"cascade" that includes: attendance at antenatal care (ANC), 
HIV counselling and testing, provision of combined antiretro- 
viral therapy (cART) - previously prophylactic antiretrovirals 
(ARVs) - safe delivery, administration of prophylactic ARVs for 
the exposed newborn through the breastfeeding period, safe 
infant feeding, infant follow-up including HIV testing, family 
planning and linkages to long-term HIV care and treatment 
[5,6]. Comprised of a sequence of linked events, the pMTCT 
cascade provides a useful framework for organizing pMTCT 
services and quantifying attrition along the pathway from 
identifying HIV-positive women, preventing HIV transmission 
to children and ensuring long-term access to HIV care and 
treatment [7]. Improving pMTCT effectiveness requires in- 
creasing the number of women and their children who 
successfully pass through the multiple, sequential steps in 
the pMTCT cascade; therefore identifying replicable ap- 
proaches to optimize the cascade is a critical priority [8]. 

Methodologies to classify and learn from differential 
performance 

Identifying high and low-performing health facilities, and the 
drivers of performance, may highlight best practices associa- 
ted with effective pMTCT service delivery in resource-limited 
settings for replication in other pMTCT systems. Two metho- 
dologies developed by systems management and behavioural 
sciences that utilize performance classification and exploit 
performance differentials to improve health system function- 
ing and individual-level health behaviours include Lot Quality 
Assurance Sampling (LQAS) and Positive Deviance [9-12]. 

LQAS includes the inspection of a small representative 
sample of a production lot, and if the number of defective 
goods in the sample exceeds a predetermined allowable 
number, the lot is rejected; otherwise the lot is classified as 
having acceptable quality [13]. 

LQAS has been applied to the health sector to conduct 
rapid assessments of service coverage, the accuracy of health 
records, logistics and supply chain performance, training pro- 
grams and the quality of facility management [10,14-16]. 
LQAS identifies low performers, so managers can target their 
support to sub-optimally performing geographic areas or 
facilities, and provide insights into the relationship between 



improvement strategies and their effectiveness. However, 
beyond identifying failing production systems in specific 
geographic or service areas, LQAS does not provide guidance 
on which components of the production system are failing 
[17] Thus, it has limited utility for complex interventions like 
pMTCT, which may require tailored interventions depending 
on which pMTCT cascade step is under-performing. 

On the other side, positive deviance focuses on high 
performers to understand and amplify their traits, practices 
and characteristics across a broader population [18]. By 
identifying successful individuals, families or groups with 
uncommon, beneficial practices (positive deviants) - and 
modelling these practices for others - positive deviance 
intends to build on internal solutions that are potentially 
more culturally appropriate and sustainable [19]. Positive 
deviance has been applied to multiple interventions in the 
areas of nutrition, hygiene, obesity prevention, health service 
utilization, breastfeeding adherence and HIV/AIDS prevention 
[20-27]. By contrasting high and low-performing individuals, 
positive deviance provides a framework for understanding and 
improving outcomes in low-performing entities [18,28]. 

Despite their targeted use of either high (positive deviance) 
or low (LQAS) performance, neither adequately articulates a 
standardized approach for overall performance classification, 
which is challenging for cascade type services that span 
multiple arenas. Various methodologies assess the perfor- 
mance and/or effectiveness of pMTCT programs using single 
measures. Some of these include the uptake of ARV regimens 
during pregnancy and delivery as measured through program 
data [29]; the use of cord blood assessments which are able to 
assess actual ARV coverage for comparison with service data 
[30]; and the use of early infant diagnosis which is arguably the 
most accurate measure of this program which is designed 
to prevent mother-to-child transmission of HIV, as well as to 
promote early clinical care and continuous follow-up pMTCT 
for the exposed infant [31]. However, for health managers 
to view their service as a system, a broader understanding 
of the pMTCT intervention is needed. Specifically, methodol- 
ogies which promote a broader vision for health workers must 
take into account the pMTCT cascade as well as specific health 
system factors that may influence its effectiveness. 

The objective of this study is to develop and explore a 
performance classification approach for health managers to 
use to identify modifiable health system factors associated 
with pMTCT performance. By describing this approach, we 
provide a novel model for pMTCT performance assessment 
that relies on routinely collected and reported data, and 
which can be used by health managers to identify modifiable 
factors associated with pMTCT performance to inform pro- 
gram improvement efforts. 

Methods 

Study design 

Activities were conducted as part of a multi-methods, cross- 
sectional study designed to identify modifiable and non- 
modifiable health systems factors associated with pMTCT 
service performance. The results presented here contributed 
to the parent study which will be conducted in three sub- 
Saharan countries by developing the health facility perfor- 
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mance ranking approach as a first step for identifying systems- 
level factors associated with high and low-performing clinics. 

Study setting 

Thirty public sector health facilities with pMTCT services in 
three districts along the Beira corridor (the main transport 
route connecting the port city of Beira with Zimbabwe) in 
Sofala province, central Mozambique, were included in the 
study. Sofala has an estimated population of 1.8 million of 
which 47% live in the study districts of Beira city, Dondo and 
Nhamatanda [32]. Sofala province has an estimated adult HIV 
prevalence of 15.5% [33], which has been consistently higher 
among women routinely tested for HIV in ANC (17.8%) [34]. 

Since its introduction in 2002, pMTCT expansion has 
increased to reach 100% of all public sector clinics with ANC 
services in the three study districts (and 86% of all facilities 
nationally) [35]. The gaps in the pMTCT cascade limit its 
effectiveness resulting in approximately 28% HIV infection rate 
in infants born to HIV-positive women in 2012 despite high 
availability of pMTCTand approximately 95% ANC attendance 
rate [36]. 

Study facilities included all public sector health facilities 
that met the inclusion criteria of: 1) location in the three 
districts; 2) provision of pMTCTat ANC in the last six months; 
and 3) consent to participate in the study. Two facilities were 
excluded because they were unwilling to participate or had 
limited access due to flooding. 

Data sources 

Clinic data for pMTCT performance ranking were sourced 
from provincial program reports covering January-December 
2012. Data were based on monthly health facility data, which 
in parallel is entered into the national health information 
system at the district level. Monthly facility-level data were 
assessed for availability by the study team to identify missing 
reports, and irregular or missing data were crosschecked with 
facility-level registries to ensure accuracy. Missing data were 
recovered for all measures except cluster of differentiation 
4 (CD4) testing data, which were inconsistently available at 
the facility-level. 

Data on health facility characteristics were collected using 
a survey developed for study purposes from November 2012 
to January 2013 (Supplementary file). Descriptive, facility- 



level variables were identified through reviews of published 
literature on pMTCT and quality improvement, and the final 
list of facility characteristics was developed in consultation 
with provincial program managers and technical advisors [37- 
39]. The data collection form was developed in Portuguese 
and piloted in one facility before study assistants visited all 30 
health facilities to collect information from facility managers 
and front line health workers. Data from each facility were 
double entered by study personnel to ensure their accuracy. 

Variable definition 

Performance ranking 

Three performance measures were a priori selected based on 
their presence in routine reporting systems and importance 
for successful pMTCT, including: 1) the proportion of pregnant 
women in ANC tested for HIV at their first visit; 2) the pro- 
portion of pregnant women with a positive HIV test at first 
ANC visit who had a CD4 test in pregnancy; and 3) the 
proportion of women with a positive HIV test in the first ANC 
visit accessing effective PPO or cART in pregnancy (Table 1). 
HIV testing in the first ANC visit was selected (versus testing at 
any time during pregnancy) because testing in the first ANC is 
standard as per Ministry of Health (MOH) guidelines, and 
because an effective pMTCT package should initiate as early 
in pregnancy as possible [40]. Single-dose nevirapine was not 
included as an effective pMTCT PPO in the third measure 
because of its relative ineffectiveness in preventing HIV 
transmission [41]. 

The 12 months of pMTCT cascade data were used to 
develop summary performance scores for each facility 
(n =30). First, each of the three indicators were estimated 
over a 12-month period (calendar year 2012). These individual 
indicators were then multiplied to generate the composite 
indicators. Facilities were ranked into three performance 
categories (high/middle/low) based on tertiles in the dis- 
tribution of performance outcomes, rather than using a 
specific performance threshold level. Visual assessment was 
then used with one measurement strategy (HIV testing and 
ARV treatment and ARV PPO) defined as the reference stra- 
tegy. This strategy was selected as the reference as it was 
composed of complete data sets and represented multiple 
steps in the pMTCT cascade. 



Table 1. Ranking measures 





Indicator 


Pros 


Cons 


1 


% of women with HIV test at first ANC Visit 


High availability and standardized 


Little heterogeneity; almost universally high masks 






collection and reporting across 


performance differences 






facilities 




2 


% of pregnant women with positive HIV test 


Measures integration across services 


Data inconsistent or unavailable; cannot link women 




at first ANC visit with a CD4 test while in 


(e.g. laboratory vs. clinic), high 


with outcome to those with HIV test at first ANC visit 




pregnancy 


variation across facilities 


using routinely reported data 


3 


% of pregnant women with a positive HIV test 


High availability and standardized 


Cannot link women with outcome to those with HIV 




at first visit who initiate either bi-/tri- 


collection and reporting across 


test at first ANC visit using routinely reported data 




prophylaxis or cART prior to delivery 


facilities 





ANC: antenatal care; cART: combination antiretroviral therapy. 
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In order to assess the value of comparing high versus 
low-performing facilities in identifying modifiable health 
facility characteristics associated with pMTCT performance, 
we recoded facilities into two additional performance groups 
- one joining middle and high performers (to compare 
with low performers), and a second joining middle and low 
performers (to compare with high performers). 

Modifiable and non-modifiable facility characteristics 
Non-modifiable facility characteristics considered included 
facility type, classified into three levels - quaternary/tertiary 
hospitals, secondary hospitals and primary health centres, as 
well as catchment population sizes which were provided by 
provincial and district authorities. Additional non-modifiable 
factors collected included geographic location which was 
defined as urban/peri-urban/rural based on their location in 
Beira, Dondo or Nhamatanda municipalities (urban), in the 
outlying neighbourhoods of Beira (peri-urban), or outside of 
Beira, Dondo, or Nhamatanda municipalities (rural) and year 
of pMTCT initiation which was provided by facility leadership. 

Modifiable facility level characteristics included staffing 
which was defined as the number of health workers of cadres 
most relevant to pMTCT service delivery (physician, non- 
physician clinician (NPC), maternal and child health nurse, 
general nurse, midwife, custodian, social worker and activist). 
Distance to a laboratory with CD4 capacity was also con- 
sidered modifiable as it could hypothetically be changed if 
CD4 capacity was introduced through new machines or 
new PIMA technology. This was estimated using driving 
distances between health facilities provided by provincial 
authorities. 

Other modifiable factors measures to describe pMTCT 
organization included integration with laboratory services 
(number of days per week with CD4 blood draws, availability 
of on-site laboratory capacity for CD4 and other laboratory 
monitoring), pharmacy services (medicines distributed via 
pharmacy or ANC/maternity) and outpatient care (adult 
and paediatric patient referral and tracking for continuity). 
Also, modifiable measures of community linkages included 
the presence of a mothers' support group, whether commu- 
nity activists carried out patient tracking and whether health 
workers performed regular community outreach. General 
management practices were measured by the frequency 
of staff meetings. A list of essential medicines, supplies and 
materials related to pMTCT service provision was included in 
the factors list to assess the availability of key items over 
the preceding three months, as well as the length of stock 
outages, and was confirmed using stock registries. These were 
all deemed modifiable as innovations could be introduced to 
improve them at the facility, district or provincial level. 

Finally overall PMTCT service utilization was captured 
through four patient volume measures over the six-months 
before data collection, including the number of ANC consults, 
the number of postpartum consults, the number of institu- 
tional births and the number of modern family planning 
methods distributed. 

Statistical analysis 

To refine the ranking procedures, we first explored whether 
the rank performance order for the 30 study facilities 



changed according to performance measurement (including 
each of the three outcome measures alone, a composite 
indicator multiplying indicators one and three for each facility 
(HIV testing and effective PPO or cART), and a composite 
indicator multiplying indicators one, two and three for each 
facility (adding CD4 testing to the previous indicator). Visual 
assessment focused on the consistency of rank order across 
high, medium and low categories depending on ranking 
strategy, using the composite indicator of HIV testing and 
effective PPO or cART as the benchmark, with the top 10 
performing sites shaded, the middle 10 in white and the 
bottom 10 dotted. 

Assessment of the impact of ranking process on facility- 
level characteristics associated with facility performance was 
carried out for three of the five ranking approaches, excluding 
CD4 testing data which were found to be less available and 
reliable. Unadjusted Chi-square tests for independence were 
performed to estimate the association between performance 
and health facility characteristics. For continuous variables, 
logistic regression for performance outcomes was used to 
quantify the magnitude and statistical significance of any 
associations with performance. 

Table 2. Characteristics of study slinics 

N % 

Clinic location 

Rural 18 60 

Urban 5 17 

Peri-urban 7 23 
Clinic type 

Quaternary/tertiary 1 3 

Secondary 1 3 

Primary 28 93 
Public/private 

Public 27 90 

Private 0 0 

Mixed 3 10 
NGO support 

None 17 57 

One 5 17 

Multiple 8 26 

Year of pMTCT initiation 

Before 2005 5 17 

Between 2006 and 2008 21 70 

After 2008 4 13 

Other characteristics Mean Median 

Distance to reference laboratory (km) 22 16 

Catchment population (people) 27,774 19,644 

No. 1st ANC visits in the last 6 months 674 468 

No. institutional births in the last 6 months 482 257 

No. post-partum visits in the last 6 months 418 189 

NGO: non-governmental organization; pMTCT: prevention of mother- 
to-child-transmission; ANC: antenatal care. 
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After describing factors associated with high versus low- 
performing pMTCT clinics, we assessed how the inclusion of 
the 10 middle-performing clinics affected the list of factors 
significantly associated with differential pMTCT performance. 
For this analysis, we developed ordinal logistic regression 
models including continuous clinic characteristics found to 
be significant in the bivariate analyses, and examined the 
magnitude of the associations between three-level (low, 
middle, high) compared to two-level (low, high, excluding the 
10 middle-performing clinics). Next, we aggregated low and 
middle-performing facilities to compare their characteristics 
to high-performing facilities, and then aggregated middle and 
high-performing facilities to compare their characteristics 
with low-performing clinics. This analysis ranked facilities 
using indicators one and three (HIV testing at first ANC and 
receipt of effective PPO or cART), as these measures were 



most available and measured multiple, essential steps for 
successful pMTCT. 

Data analysis was performed using Stata vll.2 (College 
Station, TX). 

Ethical approval 

Study procedures were approved by the Ethics Committee of 
the Mozambique MOH and were determined to be non- 
research by the University of Washington Institutional Re- 
view Board. 

Results 

The majority of health facilities were public sector, small to 
mid-sized primary health centres clustered in rural areas 
along the Beira corridor, with an average catchment popula- 
tion of approximately 28,000 inhabitants (Table 2). 



Table 3. Effect of PMTCT measurement strategy on performance ranking 



HIV testing, ART-PPO 
mother 



HIV testing, CD4 testing, 
ART-PPO mother 



ART-PPO mother 



CD4 



HIV testing 



Rank Clinic Score Rank Clinic Score 



1 


A 


0.958 


2 


B 


0.941 




C 


0.902 


i 


D 


0.864 




E 


0.863 




F 


0.862 


i 


G 


0.857 




H 


0.856 
0.853 


11 


K 


0.776 1 


12 


L 


0.761 


13 


M 


0.749 


14 


N 


0.732 


15 


0 


0.727 


16 


P 


0.726 


17 


Q 


0.714 


18 


R 


0.713 


19 


S 


0.699 


20 


T 


0.691 


21 


U 


0.671 


22 


V 


0.669 


23 


w 


0.652 


24 


X 


0.640 


25 


Y 


0.627 


26 


Z 


0.615 


27 


AA 


0.600 


28 


BB 


0.579 


29 


CC 


0.494 


30 


DD 


0.477 



9 
10 
11 
12 

13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 



M 
L 
S 
O 
P 
R 
Q 
T 
U 
V 

w 
x 

Y 

z 

AA 
BB 
CC 
DD 



Rank 



Clinic Score Rank Clinic Score Rank Clinic Score 



1 


j 


0.745 


2 


E 


0.723 


3 


A 


0.705 



5 


K 


0.620 


6 


N 


0.620 



0.601 
0.586 
0.576 
0.571 



0.554 
0.526 
0.492 
0.484 
0.484 
0.483 
0.476 
0.461 
0.447 
0.446 
0.434 
0.427 
0.418 
0.410 
0.402 
0.401 
0.329 
0.318 



10 

12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 



0.611 



K 
N 
P 
T 
R 
X 
L 
O 
S 
Q 
W 
V 
Y 

AA 
BB 
Z 

CC 
DD 
V 



0.647 

0.567 
0.561 
0.547 
0.545 
0.540 
0.524 
0.521 
0.453 
0.436 
0.429 
0.396 
0.367 
0.308 
0.304 
0.275 
0.229 
0.095 
0.000 
0.000 



1 


V 


1.00 


2 


Q 


1.00 


3 


Z 


1.00 


4 


L 


1.00 




9 


S 


0.078 


9 


K 


0.98 


10 








L 


0.056 




\j 


u.yo 


11 


BB 


0.043 












muni 


12 


S 


0.96 


13 


R 


0.024 | 








0.007 


14 




0.95 


14 


AA 


jjjjjjjjjjj 


■Hi 


mUHf 


15 


111111 


0.95 


16 


Q 


0.000 








17 


Z 


0.000 


17 






HI 




0.000 


18 


H 


0.93 


19 


CC 


0.000 


19 


G 


0.92 


20 


0 


0.000 


20 




0.91 


21 


u 


0.000 


21 


W 


0.91 


22 


w 


0.000 


22 


P 


0.91 


1 23 


P 


0.000 


23 


N 


0.90 


1 24 


T 


0.000 


24 


AA 


0.90 














25 


X 


0.000 


25 


CC 


0.89 


26 


111 1 


0.000 


26 


R 


0.89 


27 


DD 


0.000 


27 


BB 


0.88 


28 


V 


0.000 


28 


M 


0.85 






0.000 


29 


T 


0.84 


30 


D 


0.000 


30 


X 


0.76 



Dark grey: high performance; white: middle performance; spotted grey: low performance using dual HIV testing & ART-PPO mother ranking 
schema; HIV: human immunodeficiency virus; ART: a nti retroviral therapy; PPO: prophylaxis; CD4: cluster of differentiation 4. 
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Table 4. Clinic characteristics associated with high pMTCT performance by ranking approach 



Ranking measure 



+ Association 



-Association 



No association 



HIV test at first ANC Catchment size 
visit & PPO-ART for No. MCH nurses 
mother No. MDs & NPCs 

PIMA CD4 analyses at 

facility 
No. of deliveries in 
prior 6 months 



PPO-ART for mother Catchment size 

No. of MCH nurses 
No. of MDs & NPCs 
PIMA CD4 analyses at 
facility 

No. of deliveries in 
prior 6 months 
HIV+ mothers 
support group at 
facility 



HIV test at first ANC ANC stock outs of: 
visit AZT 



0.01* Wait time between PCR 

<0.01* blood draw & receipt of 

<0.02* results at facility 
<0.04 

<0.01* 



<0.01* Wait time between PCR <0.05* 

<0.01* blood draw and receipt of 

0.03* results at facility 

0.04 

<0.01* 
<0.01 



0.04 None 



0.04* Clinic location (rural, urban, peri-urban) 0.29 

NGO support 0.17 

Active tracing LTFU 0.52 

HIV+ mothers support group at 0.19 
facility 

Community linkages 0.43 

Schedule of receiving requisitions & 0.36 
delivering consumables 
ANC stock outs of: 

AZT 0.79 

HIV rapid test 0.59 
Pharmacy stock outs of: 

HIV rapid test 0.79 
Clinic location (rural, urban, peri-urban) 0.37 

NGO support 0.16 

Active tracing LTFU 0.52 

Community linkages 0.66 

Schedule of receiving requisitions & 0.57 
delivering consumables 
ANC stock outs of: 

AZT 0.79 

HIV rapid test 0.59 
Pharmacy stock outs of: 

HIV rapid test 0.38 
NA Clinic location (rural, urban, peri-urban) 0.42 

NGO support 0.55 

Active tracing LTFU 0.52 

Community linkages 0.66 

Schedule of receiving requisitions & 0.16 
delivering consumables 
ANC stock outs of: 

HIV rapid test 0.59 
Pharmacy stock outs of: 

HIV rapid test 0.79 

Catchment size 0.83* 

No. of MCH nurses 0.25* 

No. of MDs & NPCs 0.61* 

PIMA CD4 analyses at facility 0.33 

No. of deliveries in prior 6 months 0.80* 
HIV+ mothers support group at facility 0.55 

Wait time between PCR blood 0.37* 
draw and receipt of results 
at facility 



All p-values are from chi-squared tests unless noted with *(unadjusted logistic regression). 

PPO: prophylaxis; ART: a nti retroviral therapy: ANC: antenatal care; MCH: maternal child health; MD: medical doctors; NPC: non physician 
clinicians; PIMA CD4: point of care rapid CD4; NGO: non-governmental organization; LTFU: lost to follow up; AZT: Zidovudine. 



Two hospitals were in the sample, including one quatern- 
ary-level referral hospital and one rural hospital. Over half of 
the facilities reported receiving no non-governmental organi- 



zation (NGO) support, and almost 70% began pMTCT services 
between 2006 and 2008, during the push to fully extend 
pMTCT services to all health facilities with maternal child 
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health (MCH) services nationwide. The mean distance be- 
tween health facilities and their CD4 reference laboratory 
was 22 km (13.1 miles). 

Effect of performance measures on ranking 

Visual assessment of the effect of performance measures on 
ranking found a consistency among lower performing clinics, 
with more variability in rank order among higher performing 
facilities (Table 3). 

Of the three indicators representing a single step of 
the pMTCT cascade, receipt of ARV PPO or cART for women 
identified as HIV positive at first ANC visit was the most 
consistent in rank order compared with the reference 
composite indicator of HIV testing and ARV therapy and ARV 
PPO. The single indicator with the most divergence from the 
composite measure was HIV testing at first ANC visit, which 
was high at all pMTCT clinics (over 95% for half of 
the study clinics), and thus provided insufficient variability 
for characterizing high and low-performing pMTCT clinics. 
Though the proportion of women who received CD4 test- 
ing services was low (53.3%), it was included as a performance 
measurement strategy because at the time of study imple- 
mentation, CD4 testing was a prerequisite to access care. 

Impact of ranking approach on characteristics associated 
with performance 

Ranking strategy substantially impacted clinic characteristics 
associated with pMTCT performance (Table 4). 

Using the two variable performance ranking approaches 
(HIV test at first ANC visit and ARV PPO or cART for mother), 
five factors were found to be significantly associated with 
higher pMTCT performance, including larger catchment area, 
higher number of institutional deliveries, availability of PIMA 
point-of-care CD4 capacity onsite, higher numbers of MCH 
nurses and higher number of MDs and NPCs. Longer lag time 
from polymerase chain reaction (PCR) blood draw to receipt 
of results at facility was associated with poor pMTCT per- 
formance, but shorter lag times were not associated with 
high-performing pMTCT clinics. Factors not associated with 
performance included clinic location, availability of NGO sup- 



port, presence of a women's support group, active community- 
level patient tracking systems and stock-outs of essential 
testing supplies and medicines. 

Using maternal receipt of ARV PPO or cART as the single 
indicator for performance ranking, the presence of a support 
group for people living with HIV was also found to be 
significantly associated with higher performance; the remain- 
ing factors remained unchanged. 

Using a HIV test at the first ANC visit as the single indicator 
for performance ranking dramatically altered the factors 
associated with pMTCT performance, with only stock-outs of 
AZT in ANC remaining associated with high pMTCT perfor- 
mance. All other indicators that were positively or negatively 
associated with pMTCT performance were no longer sig- 
nificant in this analysis. 

Impact of middle-performers on the association of clinic 
factors and pMTCT performance 

Removing the 10 middle-performing clinics and comparing 
the magnitude of associations with three-level performance 
did not dramatically alter the results (Table 5). 

With the exception of lag time between PCR blood draw and 
receipt of results at the pMTCTclinic, all additional explanatory 
factors remained significantly associated with performance. 

Aggregating the middle-performing clinics with the high or 
low-performing groups slightly impacted the characteristics 
significantly associated with pMTCT performance. Most no- 
tably, catchment area size was less strongly associated with 
performance when middle-performing clinics were aggregated 
with the low-performing group, though the strength and 
direction of the association was similar to the analysis omitting 
middle-performing clinics. Decreased waiting time between 
PCR blood draw and receipt of results at the clinic was more 
significant when the middle-performing facilities were aggre- 
gated with the high-performing group compared with the 
analysis omitting middle-performing facilities. For both the 
number of MCH nurses and number of MDs/NPCs, there 
was no meaningful difference in direction and strength of 
association when the middle-performing clinics were included 
in the analysis. 



Table 5. Associations of continuous clinic characteristics with performance, using different performance outcomes 



Three-level Two-level performance Two-level performance Two-level performance 

performance* (mid-performers omitted) (low/middle vs. high) (low vs. middle/high) 

Odds ratio Odds ratio Odds ratio Odds ratio 

Variable (95% CI) p (95% CI) p (95% CI) p (95% CI) p 



Catchment area (per 10,000 1.92 (1.09-3.38) 0.02 3.57 (1.84-6.91) 0.0002 1.47 (0.92-2.33) 0.10 4.74 (1.46-15.36) 0.01 
inhabitants) 

Wait time (days) between PCR 0.95 (0.90-1.00) 0.04 0.10 (0.01-1.41) 0.09 0.96 (0.90-1.02) 0.15 0.91 (0.83-1.00) 0.05 
blood draw & receipt of result 
at clinic 

Number of MCH nurses 1.12 (1.03-1.22) 0.009 1.26 (1.08-1.47) 0.003 1.08 (0.90-1.02) 0.06 1.21 (1.05-1.41) 0.01 

Number of MDs & NPCs 2.21 (1.12-4.39) 0.02 5.16 (1.22-21.86) 0.03 2.00 (0.99, 4.01) 0.05 5.37 (1.31-21.99) 0.02 



PCR: polymerase chain reaction; MCH: maternal child health; MD: medical doctors; NPCs = non-physician clinicians; *OR calculated using ordinal 
logistic regression with robust standard errors. 
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Discussion 

In this article, we explore an approach to classify pMTCT 
performance based on routinely collected health systems 
data, coupled with additional data collection on modifiable 
and non-modifiable health systems factors, to identify best 
practices for pMTCT. We found that using two measures to 
classify pMTCT performance was preferable to one, though 
one measure (effective PPO or cART in pregnancy) provided 
sufficient performance variability to detect associations with 
health system factors. In general, combining critical individual 
components into a single composite measure - as part of a 
dashboard of measures - may facilitate system-level changes 
by highlighting the need for better care coordination and 
accountability across multiple providers or services [42]. 
However, the use of composite metrics in settings with weak 
health information systems must ensure that individual 
measures within the composite are based on the strongest 
evidence available [43]. 

Classifying high and low performance provided consistent 
results across ranking measures, though granularity was 
improved by including middle performers in either the high 
or low-performing groups. In this analysis, human resource 
availability (especially MCH nurses), catchment size and 
utilization were positively associated with effective pMTCT 
service delivery. A delay in PCR results was negatively 
associated with pMTCT service delivery. Notably, measures 
of community linkages were not associated with pMTCT 
service delivery across the ranking approaches. 

This study was limited by its relatively small sample of 30 
facilities from one geographic location in one country, which 
affects the generalizability of the findings. Nonetheless, the 
methodology of using high and low performance shows 
promise, providing a framework that can be evaluated in 
other settings for HIV and non-HIV related services, and can 
be used to verify and add to the initial list of modifiable and 
non-modifiable factors associated with pMTCT performance. 

There are considerable on-going efforts to evaluate pMTCT 
programs at the global, country, and community level. These 
include novel techniques to estimate HIV-free survival using 
community surveys as well as more traditional health facility 
level surveys to assess the relationship between facility 
characteristics and pMTCT coverage across multiple countries 
[44,45]. The methodology explored in this paper is meant to 
complement this existing work by providing health managers 
with a practical way in which to benchmark quality improve- 
ment efforts. 

The principal strength of this study is its reliance on 
routine health information system data coupled with simple 
data collection that is feasible during routine supervision. 
By using health facilities as the unit of analysis (rather than 
individual patient attributes, knowledge, or behaviours), 
this study delves into broader determinants of effective 
pMTCT services that are under the control of health system 
managers, and can be modified to lead to broad improve- 
ments in service effectiveness. Furthermore, the simple 
analysis approach is geared towards district MCH managers 
to improve data use for pMTCT service redesign and en- 



hancement, relying on strategies that are effective in their 
service delivery context. 

In this study, increased human resources were significantly 
associated with pMTCT performance, which is consistent with 
literature that has highlighted the importance of human 
resources for better health systems outputs and measures of 
population health impact [46,47]. This relationship is logical 
given severe personnel constraints, and in the context of 
improving pMTCT service quality, underscores the need for 
more efficient allocation of limited human resources and 
significant investments to produce and retain human re- 
sources. Thus, any intervention to improve pMTCT delivery 
must address human resource constraints, either directly, 
or more likely, indirectly through the introduction of systems 
analysis and improvement approaches to increase the effi- 
ciency of delivery. The lack of associations between pMTCT 
performance and community and social support may be the 
result of the sample size, or the overwhelming importance of 
other factors like personnel availability. 

The inability of individual measures (especially HIV testing 
in pregnancy) to capture a "high-performing" facility was 
likely due to insufficient variability in this measure across 
study facilities. Further research into the variability of pMTCT 
cascade measures in other settings is warranted to guide 
application of the ranking approach. 

The use of routine health information system data to 
classify health facility pMTCT performance represents an 
opportunity and a challenge in low resource, high-burden 
settings such as Mozambique. Using existing data sources 
reinforces routine information systems and can lead to 
investment by health workers in assessing and improving 
service delivery, though incomplete or inconsistent data 
can result in inconsistent rankings (such as CD4 data, which 
incidentally will become a less important gateway-to-care 
measure in decisions about patient care as Option B+ is 
introduced). 

Conclusions 

This study supports using fewer, targeted performance 
measures for systems analysis. As reporting requirements 
increase to meet the expectations of funders, less time is 
available to use data for program improvements by district 
and frontline health managers. Targeting high and low 
performance is one practical approach to rapidly use routine 
pMTCT cascade data to achieve a systems perspective of 
pMTCT functioning at the health facility level, and to exploit 
differences across facilities to identify best practices across 
health facilities to inform system improvement efforts. 
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